Vertex AI pipelines offer a handy way to implement end-to-end ML workflows from data collection to endpoint monitoring with extremely low effort. For new users, the easiness of development and deployment is largely thanks to the Vertex AI pipeline example offered by GCP. Despite the comprehensive demonstration of the essential components, the official example also exposes the feasibility for users to customize and enhance the pipeline based on their own needs. Amongst all, one of the most exciting components is the distributed Hyperparameter Tuning (HPT) that is capable of exploring a huge search space to identify the best hyperparameters in a short time. However, the limitation of the tutorials is that the distributed HPT is presented as a standalone HPT task/pipeline and it doesn't explicitly present the approach to integrate into the existing Vertex AI pipeline shown in the Vertex AI pipeline example, which motivates me to share my successful attempt that bridges the gap. I believe this will benefit many businesses who have built or will build their ML workflows based on Vertex AI pipeline.

We describe the new BigQuery and BigQuery ML (BQML) components now available for Vertex AI Pipelines, enabling data scientists and ML engineers to orchestrate and automate any BigQuery and BigQuery ML functions. We also showed an end-to-end example of using the components for demand forecasting involving BigQuery ML and Vertex AI Pipelines.

PyTorch is an open source machine learning framework, primarily developed by Meta (previously Facebook). PyTorch is extensively used in the research space and in recent years it has gained immense traction in the industry due to its ease of use and deployment. Vertex AI, a fully managed end-to-end data science and machine learning platform on Google Cloud, has first class support for PyTorch making it optimized, compatibility tested and ready to deploy. We started a new blog series - PyTorch on Google Cloud - to uncover, demonstrate and share how to build, train and deploy PyTorch models at scale on Cloud AI Infrastructure using GPUs and TPUs on Vertex AI, and how to create reproducible machine learning pipelines on Google Cloud . This blog post is the home page to the series with links to the existing and upcoming posts for the readers to refer to.